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Abstract Body 



Background: 

Being able to analyze complex ecosystems is fundamental for becoming ecologically- 
literate citizens (Jordan, Singer, Vaughan, & Berkowitz, 2009; Sabelli, 2006). Like all complex 
systems, making sense of ecosystems is challenging because they transcend spatial, temporal, 
and cognitive boundaries (Pickett, et al, 1997) and necessitate understanding how different 
components, mechanisms and phenomena are interconnected (Covitt, Gunckel, & Anderson, 
2009; Jacobson & Wilensky, 2006; Jordan et al., 2009; Mohan, Chen, & Anderson, 2009). 
Furthermore, complex systems are comprised of multiple interrelated levels that are dynamically 
related, making it difficult even for experts to understand and to predict (Hmelo-Silver & 
Azevedo, 2006; Jacobson & Wilensky, 2006; Simon, 1996). 

A range of studies have demonstrated that it is particularly challenging for learners to 
grasp relationships within systems (Ben-Zvi Assaraf & Orion, 2005; Gallegos et al 1994; Penner, 
2000). Often, learners focus on simple linear relationships and visible components of an 
ecosystem (Hmelo-Silver, Marathe, & Liu, 2007; Hogan, 2000; Hogan & Fisherkeller, 1996; 
Leach et al. 1996; Reiner & Eilam, 2001). In clinical interviews, when novices were asked to 
identify features of an aquarium system, they tended to emphasize visible components, such as 
fish and rocks, and rarely mentioned invisible components, such as oxygen, nitrogen, and 
bacteria (Hmelo-Silver, Marathe, & Liu 2007; Hmelo-Silver & Pfeffer, 2004). Other research has 
found that student explanations favor single causal and linear connections between system 
components (Grotzer & Basca, 2003). 

A promising approach for promoting systems thinking in a way that can enable students 
to think about multiple interacting components and their fates is encouraging Structure- 
Behavior-Function (SBF) thinking (Goel et al. 1996; Hmelo-Silver et al. 2007; Goel, Rugaber & 
Vattam, 2009). Artificial Intelligence (Al) theories of model-based analogies in engineering 
design have led to both the methodology and technology for building models of physical systems 
through Structure-Behavior-Function (SBF) modeling (Bhatta & Goel 1997; Chandrasekaran, 
Goel & Iwasaki 1993; Goel 1991, 1996; Goel & Batta, 2004; Goel & Chandrasekaran 1989). An 
SBF model of a system explicitly represents its structure [S] (i.e., its configuration of 
components and connections), its functions [F] (i.e., its output), and its behaviors [B] (i.e. its 
internal causal processes that enable the functions of the components into the functions of the 
system). 

Empirical research has demonstrated that experts represent complex systems in terms of 
interrelated structures, behaviors and functions, whereas novice understanding is characterized 
primarily by identifying isolated structures, demonstrating minimal understanding of functions, 
and largely missing system behaviors (e.g., Hmelo-Silver, Marathe, & Liu, 2007; Hmelo-Silver 
& Pfeffer 2004). Based on this research, we hypothesize that using SBF to guide conceptual 
representations provides a deep principle that should help students understand multiple complex 
systems (Hmelo-Silver et al., 2007; Goldstone & Sakamoto, 2003). 

Here we describe a study for which we enacted a two-week aquarium unit. The 
technology consisted of a suite of computer tools: a function-oriented hypermedia (Liu & 
Hmelo-Silver, 2009), simulations of macro- and micro-level processes (Liu & Hmelo-Silver, 
2008; Gray et al. 2008), and the Ecosystem Modeling Toolkit (EMT, building on earlier work 
with the Aquarium Construction Toolkit; Vattam et al., 2011). 
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Purpose / Objective / Research Question / Focus of Study: 

In this research, we present both the design and preliminary testing of a technology- 
intensive classroom intervention designed to support middle schools students’ understanding of 
an aquatic ecosystem. The goals of our intervention are to help learners develop deep 
understanding of ecosystems and to use tools that make the relationships between a system’s 
structures, behaviors, and functions explicit. 

Setting: 

The research reported here comes from data in three public middle schools in suburban 
school districts in the Northeastern United States. Four teachers used the instructional 
intervention was conducted during regular science classes. 

Participants: 

All students who returned consents in the three schools were included in the study. A 
total of 3 1 1 students completed both the pre and post test. 

Intervention / Program / Practice: 

An important goal of our instructional design is to provide learners with opportunities to 
engage with ecosystems phenomena, particularly those that are not available to their unaided 
perception. Learners find many ecosystem phenomena hard to understand because they have not 
had experiences thinking about those processes that are dynamic and outside their perceptual 
range (Jacobson & Wilensky, 2006). The SBF conceptual representation also provides a scaffold 
for overall knowledge organization because it helps learners consider the relationships among 
form and function as well as the causal behaviors and mechanisms. We make SBF explicit 
through the use of hypermedia, organized in terms of SBF (Please insert Figure 1 here), through 
NetLogo simulations that make behaviors visible (Please include Figure 2a and b here) and 
through the ACT tool (Please include Figure 3a and b), which makes SBF explicit as students 
build models using the language of the SBF conceptual representation. 

Along with the hypermedia and ACT tools, students also used NetLogo simulations to 
learn about the behaviors and functions in an ecosystem (Hmelo-Silver et al., 2007; Wilensky & 
Reisman, 2006). Using these simulations, (Figure 2) students had opportunities to explore factors 
that would affect the dynamic balance in the aquarium. For example, the macro fish spawn 
simulation allowed students to manipulate different aspects of the system such as initial 
population, spawning, filtration level, and amount of food. Thus if the students overfed the fish, 
then the increasing ammonia (from fish excretion) in the water would affect water quality and 
have toxic effects on the fish, leading to mortality. This helped problematize water quality, 
which is a black box in the macro simulation. This created the need for students to identify some 
of the invisible components within an ecosystem. For example, using the micro-level simulation, 
students could observe how crucial the nitrification cycle is for the overall health of an 
ecosystem and understand the important role that bacteria play in converting toxic forms of 
nitrogen (ammonia) into less toxic forms of nitrogen. 
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Although there was some minor variation among the four participating teachers, they 
followed the same general sequence. The science teachers introduced the unit by asking students 
to articulate their ideas about ecosystem functions, activating their prior knowledge and 
providing formative assessment for the teachers. The teachers then moved on to the ACT 
modeling tool and asked the students to represent their thoughts about ecosystems as structures 
behaviors and functions. The students recorded their ideas in a table within the ACT tool (Please 
insert Figure 4 about here). The teacher also encouraged the students to use the hypermedia to 
build on their ideas about the ecosystems. The teacher then had students explore the NetLogo 
simulations. In the simulations students, students could manipulate various ecosystem 
components (e.g., number of fish, amount of food, number of plants) in order to maintain a 
healthy ecosystem (Eberbach & Hmelo-Silver, 2010). The students worked in groups and had 
opportunities to refine their models. At the completion of the two-week period, students 
presented their models to the rest of the class. 

Research Design: 

A pre-post test single-group design was used for this study. 

Data Collection and Analysis: 

Students completed tests before and after the intervention. In completing this assessment 
instalment, students drew components of an aquatic ecosystem and were asked to show 
relationships between these components. In addition, students answered open-ended questions 
about different parts and processes of an aquatic ecosystem. They also solved problems related to 
ecosystems. 

The scoring criteria for the pre and post tests are summarized in Table 1. All of the questions 
(17) were coded based on two different scoring schemes. The first examined student 
explanations of relationships between structures and their related behaviors and functions. The 
codes were assigned to the responses on a four-point scale, shown in the upper part of Table 1 
(please insert Table 1 here). 

We also coded for whether the students were able to identify and explain relationships 
between micro and macro elements within an ecosystem. Only eight of the 17 questions were 
coded for Macro and Micro (MM) level because only these questions afforded opportunities to 
explain both micro and macro level connections. The other questions on the assessment were 
specific to either macro or micro elements within an ecosystem. The micro-macro relationship 
score was assigned as shown in the lower part of Table 1. 

The following student response on the importance of ‘waste’ to the aquatic ecosystem 
illustrates how these scoring schemes were applied. The student wrote: 

Waste is normally produced by organisms such as fish. It contains ammonia. 

Through the nitrogen cycle, bacteria breaks it down into nitrite then nitrate (which 

is a less toxic form of nitrogen), which is then used for plant growth. 

The response indicates the presence of multiple structures, such as fish, ammonia, bacteria, 
nitrites and nitrates. We considered “waste” as a stnicture; we coded “bacteria breaks it down” as 
behavior and “which is then used for plant growth” as its function. We assigned this response an 
SBF relation score of 4 as the student has identified at least one stnicture in relation to behaviors 
and functions. In addition, we assigned this response the maximum score of 3 for the micro- 



SREE Fall 2011 Conference Abstract Template 



3 




macro coding it as reflects connecting macro (waste) and micro (ammonia, nitrogen cycle) level 
structures and processes. Interrater reliability was calculated by having two independent raters 
code 20% of the sample. The overall reliability was 87% agreement. 

Findings / Results: 

The results, shown in Table 2 (please insert Table 2 here), demonstrate moderate to large 
effects of our technology-rich curriculum intervention. These results demonstrated significant 
improvement in understanding SBF relationships over time (F(l, 310) = 69.58, p < .001). In 
addition to the SBF relationships, we also measured the relationships between macro and micro 
elements of the system. Descriptive statistics are shown in Table 2. The results from a repeated 
measure ANOVA found a significant gain from pre to post test, F (1, 310) = 193.30 p < .001). 

Conclusions: 

In summary, students significantly improved their understanding of the aquarium 
ecosystem in terms of relationships between structures, functions, and behaviors. In addition, 
students increased their identification of micro level structures and there was a trend toward 
students’ articulating relationships between micro to macro level structures. It is the increase in 
the discussion of relationships that we argue are necessary for a more sophisticated 
understanding of ecosystems. 

From an ecosystem perspective, the interrelationships between structural and 
behavioral/functional levels represent mechanistic explanations of ecological phenomena. A 
critical aspect of these mechanistic explanations is drawing links that are not solely linear or one- 
to-one. Again, previous study has found that students have a tendency to pose linear relations 
when representing ecosystems (e.g., Hogan 2000; Hogan & Fisherkeller 1996; Leach et al. 1996; 
Reiner & Eilam 2001). Perhaps the SBF conceptual tool enabled students to conceptualize at a 
more general level enabling more links to be made beyond lines. 

Relating the invisible components in complex systems is something that middle school 
students find difficult (Liu & Hmelo-Silver 2009). By thinking about elements within a system 
using the SBF framework, micro level phenomena might be made more salient. In addition, SBF 
thinking provides a language by which students can both think about and describe the levels 
within the complex ecosystem. 

A limitation of this research is the pre -post design with no comparison group. This 
design was chosen as a first step during iterative development as part of a program of design 
research. Additional analyses are being conducted to better understand achievement among 
different subgroups. In addition, we are in the process of qualitative analysis of both written and 
video data to explore how student develop their understanding of complex systems and how the 
technology serves to mediate student inquiry. 

Our findings have implications for instruction. The SBF conceptual framework provides 
students with a language to express notions about complex ecosystems. Given that students have 
difficulty transferring ideas between one ecosystem to another, the SBF language might serve to 
broaden the scope of ideas a student has which, might promote abstraction of ecosystem 
concepts. Future directions should include design of curricula that feature other natural systems 
in an effort to understand how students transfer ideas. Given the value of systems understanding 
to ecological literacy, these investigations will be important in furthering our knowledge of how 
to improve teaching and learning of complex scientific phenomena. 
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Figure 1 . Function-centered hypermedia 
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Figure 2. NetLogo Fish Spawn and Nitrification simulations 
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Aquarium Construction Toolkit 3.0 
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Figure 3a. ACT: A space to create models 




Figure 3b. ACT: Example of model created by a student. 
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Aquarium Construction Toolkit 3.0 
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Figure 4. ACT table where students record ideas as structure, behavior, and function 
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Table 1. Scoring criteria for pre and post test 



SBF Relation 


Explanation 


Score 


No Answer 




0 


S 


Identifies structure without connecting to other 
structures, behaviors, or functions. Ex: “An 
aquarium has fish, gravel, and bacteria.” Ex: A 
drawing with no connections (written or drawn). 


1 


S:S 


Identifies some relationship between structures. Ex: 
“Bacteria are in the gravel.” Ex: A drawing with 
connections but no elaboration (written or drawn). 


2 


S:B or S:F 


Identifies structures in relation to behaviors or 
functions. Ex: (B) “Fish eat the food.” (F) “Fish get 
energy.” Ex: A drawing with connections and 
elaboration (written or drawn). 


3 


S:B:F 


Identifies structures in relation to behaviors and 
functions. Ex: “The fish eats food to get energy.” 
Considerations: 

-Children may include many individual SB’s and 
SF’s, but to code an answer as SBF, the all three 
must reflect some relationship to each other. 

-SBF thinking is not necessarily represented in one 
sentence as the example here. 


4 


Micro/Macro 

Level 


Explanation 


Score 


No Answer 




0 


Macro or 
Micro 


Identifies only macro or only micro structures or 
processes. 


1 


Macro + Micro 


Identifies both macro and micro structures or 
processes. 


2 


Macro ^ 
Micro 


Identifies some relationship between macro and 
micro structures or processes. 


3 
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Table 2. Descriptive statistics for pre and post tests (All n= 3 11) 



Measure 


Pretest 


Posttest 


d 


Range 


SBF relationships 


44. 64(16.17) 


54.06 (21.40) 


0.58 


0-98 


Macro/ Micro Level 


13.87 (5.12) 


19.70 (8.11) 


1.13 


0-44 
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